While software systems continue to grow in size and complexity, business demands continue to require shorter development cycles. This has led software developers to compromise on functionality, time to market, and quality of software products. Furthermore, the increased schedule pressures and limited availability of resources and skilled labor can lead to problems such as incomplete design of software products, inefficient testing, poor quality, high development and maintenance costs, and the like. This may lead to poor customer satisfaction and a loss of market share for companies developing software.
To improve product quality many organizations devote an increasing share of their resources to testing and identifying problem areas related to software and the process of software development. Accordingly, it is not unusual to include a quality assurance team in software development projects to identify defects in the software product during and after development of a software product. By identifying and resolving defects before marketing the product to customers, software developers can assure customers of the reliability of their products, and reduce the occurrence of post-sale software fixes such as patches and upgrades which may frustrate their customers.
Testing and identifying problem areas related to software development may occur at different points or stages in a software development lifecycle. For example, a general software development lifecycle includes a high level requirements/design review, a detailed requirements/design review, code inspection, unit test, system test, system integration test (SIT), potentially a performance test, and typically, a user acceptance test. Moreover, as the software development lifecycle proceeds from high level requirements/design review to, for example, system integration test (SIT), performance test and user acceptance test, costs for detecting and remedying software defects generally increases, e.g., exponentially.
As such, software developers may seek to detect and remedy software defects as early in the software development lifecycle as practical in an effort to avoid the increased risks and costs of detecting and remedying these software defects later in the software development lifecycle. To aid in detecting these software defects, an organization may utilize historical defect data for a project (e.g., a software code project) in order to project future defect patterns and trends for the project.
Currently across the industry, system integration testing (SIT) is frequently performed in an ad-hoc way by running several use cases selected by human intuition rather than via a repeatable, disciplined approach. Thus, two critical factors are not typically adequately addressed in the SIT approach: (1) system architecture (including connections and interactions); and (2) an ability to empirically determine the optimal allocation of effort across different testing focuses within the system integration testing (SIT) activity. There are no industry wide models available to provide appropriate expected distributions of defects uncovered in System Integration Testing (SIT).
As a result, different testing focus areas that an effective SIT should include are rarely if ever distributed optimally. For example, many use cases actually walk through the same connection among systems/components with the same interface and data, which produces redundant test cases. Therefore, the SIT phase of the software development life cycle tends to be one of the most expensive kinds of testing relative to the benefit received. At the same time, SIT can be the most critical testing phase to ensure a successful move to production for complex system integration projects.
Accordingly, there exists a need in the art to overcome the deficiencies and limitations described herein above.